Image pickup apparatus and control method therefor

ABSTRACT

An image pickup apparatus arranged so that an optical image of an object obtained by emitting a flash is converted into an electric signal to obtain a first image regarding photographing of the object, the same optical image of the object is converted into an electric signal to obtain a second image, a distance from the object is measured for each image area of the first image on the basis of the second image, and the first image is corrected on the basis of the distance from the object measured for each image area and light distribution characteristic data of a flash stored beforehand corresponding to the distance of the object.

This is a continuation of U.S. patent application Ser. No. 12/863,785,filed Jul 21, 2010, which is a national stage of PCT/JP2009/055550,filed Mar 13, 2009.

TECHNICAL FIELD

The present invention relates to an image pickup apparatus which picksup an image of an object by using a strobe, and a control methodtherefor.

BACKGROUND ART

With an image pickup apparatus such as a digital camera, strobephotographing has been conventionally carried out, which irradiates anobject with a flash by using a strobe which is so-called a flash unit,to thereby photograph the object.

In the strobe photographing, generally, because of strobe lightdistribution characteristics, a flash is not uniformly distributed ontothe object. Therefore, the object may look brighter at its center anddarker at its periphery, or the object may look brighter as it isnearer, and darker as it is farther because it is unreachable by theflash.

Conventionally available techniques regarding strobe photographing are,for example, disclosed in Japanese Patent Application Laid-Open Nos.H11-331575, 2003-283922, and 2005-354167. Specifically, Japanese PatentApplication Laid-Open No. H11-331575 discusses an image processingtechnique of obtaining, in portrait photographing with a wall set as abackground, a distance of a person as a main object from information ofthe object, and correcting brightness based on light distributioncharacteristics in strobe photographing. Japanese Patent ApplicationLaid-Open No. 2003-283922 discusses a technique of correcting an imagebased on light distribution characteristics including lenscharacteristics in strobe photographing. Japanese Patent ApplicationLaid-Open No. 2005-354167 discusses a technique of correcting an imageaccording to light distribution characteristics dependent on an objectdistance on the basis of pieces of distance measurement information ofsome places in strobe photographing.

However, the conventional techniques discussed in Japanese PatentLaid-Open Nos. H11-331575, 2003-283922, and 2005-354167 have haddifficulties of removing local light distribution unevenness caused bythe strobe (flash unit), from an image photographed by the strobephotographing.

DISCLOSURE OF THE INVENTION

To solve the problem, the present invention provides a technique ofremoving local light distribution unevenness caused by a flash unit,from an image photographed by using the flash unit.

According to an aspect of the present invention, an image pickupapparatus comprises a flash unit which emits a flash to an object; afirst image obtaining unit which converts an optical image of the objectformed by an optical unit into an electric signal to obtain a firstimage regarding photographing of the object; a second image obtainingunit which inverts the optical image of the object into an electricsignal to obtain a second image for measurement of distance distributionof the object; a distance measurement unit which measures a distancefrom the object for each image area of the first image on the basis ofthe second image; a storing unit which stores light distributioncharacteristic data regarding light distribution characteristics of theflash unit; and a correction unit which corrects the first image on thebasis of the distance from the object measured by the distancemeasurement unit for each image area and the light distributioncharacteristic data corresponding to the distance of the object.

According to another aspect of the present invention, a control methodfor an image pickup apparatus including a flash unit for emitting aflash to an object and a storing unit for storing light distributioncharacteristic data regarding light distribution characteristics of theflash unit, comprises: a first image obtaining step of converting anoptical image of the object formed by an optical unit into an electricsignal to obtain a first image regarding photographing of the object; asecond image obtaining step of converting the optical image of theobject into an electric signal to obtain a second image for measurementof distance distribution of the object; a distance measurement step ofmeasuring a distance from the object for each image area of the firstimage on the basis of the second image; and a correction step ofcorrecting the first image on the basis of the distance from the objectmeasured in the distance measurement step for each image area and thelight distribution characteristic data corresponding to the distance ofthe object.

Further features of the present invention become apparent from thefollowing description of an exemplary embodiment with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of an internalconfiguration of a digital camera (image pickup apparatus) according toan embodiment of the present invention.

FIG. 2 is a flowchart illustrating an example of a processing procedurein a control method for the digital camera (image pickup apparatus)according to the embodiment of the present invention.

FIGS. 3A, 3B and 3C are schematic diagrams each illustrating an exampleof a basic pixel array of a general image pickup element.

FIGS. 4A and 4B are schematic diagrams each illustrating an example of abasic pixel array of an image pickup element according to the embodimentof the present invention.

FIGS. 5A and 5B are schematic diagrams each illustrating an example of apixel structure of an image sensor (color pixel) of the image pickupelement according to the embodiment of the present invention.

FIGS. 6A and 6B are schematic diagrams each illustrating an example of apixel structure of the image sensor (color pixel) of the image pickupelement according to the embodiment of the present invention.

FIG. 7 is a schematic diagram illustrating an example of a pixel arrayof the image pickup element according to the embodiment of the presentinvention.

FIGS. 8A and 8B are schematic diagrams each illustrating an example of apixel structure of a distance measurement pixel S1 illustrated in FIG.7.

FIGS. 9A and 9B are schematic diagrams each illustrating an example of apixel structure of a distance measurement pixel S2 illustrated in FIG.7.

FIGS. 10A and 10B are conceptual diagrams of detection of imagedeviation caused by an out-of-focus state of the image pickup elementaccording to the embodiment of the present invention.

FIG. 11 is a schematic diagram illustrating a modified example of apixel array of the image pickup element according to the embodiment ofthe present invention.

FIG. 12 is a schematic diagram illustrating a modified example of apixel array of the image pickup element according to the embodiment ofthe present invention.

FIG. 13 is a schematic diagram illustrating a modified example of apixel array of the image pickup element according to the embodiment ofthe present invention.

FIG. 14 is a conceptual diagram of focus detection of the digital camera(image pickup apparatus) according to the embodiment of the presentinvention.

FIG. 15 is a conceptual diagram of focus detection of the digital camera(image pickup apparatus) according to the embodiment of the presentinvention.

FIG. 16 is a schematic diagram illustrating a status of pixel signalsread from row groups respectively including the distance measurementpixels S1 and S2 of the image pickup element according to the embodimentof the present invention.

FIGS. 17A and 17B are schematic diagrams each illustrating an example ofa pixel structure of a distance measurement pixel of the image pickupelement according to the embodiment of the present invention.

FIGS. 18A and 18B are schematic diagrams illustrating light-shieldinglayers in the distance measurement pixels S1 and S2 of the image pickupelement according to the embodiment of the present invention.

FIG. 19 is a schematic diagram illustrating an example of a pixel arrayof the image pickup element according to the embodiment of the presentinvention.

FIGS. 20A and 20B are schematic diagrams each illustrating an example ofa distance measurement pixel of the image pickup element according tothe embodiment of the present invention.

FIG. 21 is a schematic diagram illustrating a status of strobephotographing of an object which uses the digital camera (image pickupapparatus) according to the embodiment of the present invention.

FIG. 22 is a schematic diagram illustrating an example of a photographedimage when the strobe photographing of the object illustrated in FIG. 21is carried out.

FIG. 23 is a schematic diagram illustrating an example of objectdistance distribution in the strobe photographing of the objectillustrated in FIG. 21.

FIGS. 24A, 24B, 24C, 24D and 24E are schematic diagrams illustratingstrobe light distribution characteristics.

FIGS. 25A and 25B are schematic diagrams each illustrating an example ofstrobe light distribution characteristics based on a zoom position.

FIG. 26 is a schematic diagram illustrating an example of strobe lightdistribution characteristics.

FIG. 27 is a schematic diagram illustrating an example of a correctiongain in the case of the strobe light distribution characteristicsillustrated in FIG. 26.

FIG. 28 is a schematic diagram illustrating an example of a photographedimage when photographing of the object illustrated in FIG. 21 is carriedout.

FIG. 29 is a schematic diagram illustrating an example of objectdistance distribution of a horizontal image portion illustrated in FIG.28.

FIG. 30 is a schematic diagram illustrating an example of a correctiongain in the horizontal image portion illustrated in FIG. 28.

FIG. 31 is a schematic diagram illustrating an example of a photographedimage after correction processing performed for the photographed imageillustrated in FIG. 22.

FIG. 32 is a schematic diagram illustrating an example of a degree ofin-focus when an iris aperture is wide.

FIG. 33 is a schematic diagram illustrating an example of a degree ofin-focus when the iris aperture is narrow.

FIG. 34 is a schematic diagram illustrating an example of a degree ofin-focus for the horizontal image portion illustrated in FIG. 28.

FIG. 35 is a flowchart illustrating an example of a detailed processingprocedure in image correction processing of Step S6 illustrated in FIG.2.

FIG. 36 is a schematic diagram illustrating an example of objectdistance distribution in the horizontal image portion illustrated inFIG. 28.

FIG. 37 is a schematic diagram illustrating an example of a degree ofin-focus in the horizontal image portion illustrated in FIG. 28.

FIG. 38 is a schematic diagram illustrating an example of a menu settingscreen regarding correction processing in strobe light distributioncharacteristics.

FIG. 39 is a schematic diagram illustrating an example of a menu settingscreen regarding the correction processing in the strobe lightdistribution characteristics.

FIG. 40 is a flowchart illustrating an example of proceeding processingto the flowchart of FIG. 2 when various settings are made on the menusetting screens illustrated in FIGS. 38 and 39.

BEST MODE FOR CARRYING OUT THE INVENTION

A preferred embodiment of the present invention will now be described indetail in accordance with the accompanying drawings.

An embodiment of the present invention is described below by way of anexample where a digital camera is applied as an image pickup apparatusof the present invention.

FIG. 1 is a block diagram illustrating an example of an internalconfiguration of the digital camera (image pickup apparatus) accordingto the embodiment of the present invention.

A digital camera 100 of this embodiment includes an iris 101, aphotographing lens 102, an image pickup element 103, an A/D conversionunit 104, a development processing unit 105, a pupil-divided image phasedifference distance measurement unit 106, a lens control unit 107, anexposure control unit 108, and an iris control unit 109. The digitalcamera 100 of this embodiment further includes a strobe control unit110, a strobe 111, a strobe light distribution characteristic storingunit 112, a correction unit 113, an operation input unit 114, acompression processing unit 115, and an image data storing unit 116. Thecorrection unit 113 includes a correction gain calculation unit 113 aand a correction processing unit 113 b.

The iris 101 adjusts an amount of light to be guided into the digitalcamera 100, and its aperture diameter can be changed based onphotographing conditions. This iris 101 is controlled by the iriscontrol unit 109.

The photographing lens 102 has a function of guiding an optical image ofan object to the image pickup element 103, and includes one or aplurality of lenses including a focus lens. The photographing lens 102is controlled by the lens control unit 107.

The image pickup element 103 includes pixels two-dimensionally arrangedon the same plane. In each pixel, the image pickup element 103 convertsthe optical image (object image) of the object guided by thephotographing lens 102 into an electric signal (image signal) of ananalog signal to pick up an image of the object. The image pickupelement 103 includes, for example, a CCD sensor or a CMOS sensor.

The A/D conversion unit 104 converts the image signal of the analogsignal output from the image pickup element 103 into an image signal ofa digital signal.

The development processing unit 105 converts the image signal outputfrom the A/D conversion unit 104 from an RGB signal into a YUV signal toperform development processing.

The pupil-divided image phase difference distance measurement unit 106performs processing of measuring a distance from the object for eachimage area of the photographed image based on a pixel signal from adistance measurement pixel disposed in the image pickup element 103.Specifically, the pupil-divided image phase difference distancemeasurement unit 106 extracts the pixel signal from the distancemeasurement pixel disposed in the image pickup element 103, from anoutput of the A/D conversion unit 104, and obtains object distancedistribution of an image photographed within an angle of view based onpupil-divided images A and B.

The lens control unit 107 controls a focus lens of the photographinglens 102 according to distance measurement information obtained by thepupil-divided image phase difference distance measurement unit 106.

The exposure control unit 108 determines photographing conditions toenable photographing with appropriate exposure, based on the imagesignal output from the A/D conversion unit 104.

The iris control unit 109 controls the iris 101 according to thephotographing conditions (exposure conditions) determined by theexposure control unit 108.

The strobe control unit 110 controls the strobe 111 according to thephotographing conditions (exposure conditions) determined by theexposure control unit 108.

The strobe 111 emits a flash to the object when necessary due tounderexposure or the like under control of the strobe control unit 110.

The strobe light distribution characteristic storing unit 112 storesstrobe light distribution characteristic data indicating lightdistribution characteristics of the strobe 111 characterized by zoom andfocus positions of the photographing lens 102, an iris value of the iris101, or an object distance.

The correction unit 113 performs correction processing of thephotographed image on the basis of a distance of the object in eachimage area of the photographed image measured by the pupil-divided imagephase difference distance measurement unit 106, and the lightdistribution characteristic data stored in the strobe light distributioncharacteristic storing unit 112.

The correction gain calculation unit 113 a of the correction unit 113calculates a gain when the photographed image data is corrected based onthe zoom and focus positions of the photographing lens 102, the objectdistance distribution of the image photographed within the angle ofview, in-focus conditions based on the iris value of the iris 101, andthe strobe light distribution characteristic data.

The correction processing unit 113 b of the correction unit 113 performscorrection processing of the photographed image data on the basis of thegain calculated by the correction gain calculation unit 113 a.

The operation input unit 114 enters input information entered by a userto the correction unit 113. For example, the operation input unit 114includes a display screen for displaying a menu setting screen, andenters input information to the correction unit 113 via the menu settingscreen.

The compression processing unit 115 performs processing of compressingthe photographed image data corrected by the correction unit 113.

The image data storing unit 116 stores the image data compressed by thecompression processing unit 115.

FIG. 2 is a flowchart illustrating an example of a processing procedureof a control method for the digital camera (image pickup apparatus)according to the embodiment of the present invention. Specifically, FIG.2 mainly illustrates a processing procedure regarding image correctionduring strobe photographing.

First, in Step S1, for example, the correction unit 113 determineswhether to carry out strobe photographing, on the basis of inputinformation entered from the operation input unit 114. If a result ofthe determination indicates that strobe photographing is not carried out(S1/NO), the processing of the flowchart is finished.

On the other hand, if a determination result of Step S1 indicates thatstrobe photographing is carried out (S1/YES), the strobe 111 emits aflash to an object to photograph the object by the photographing lens102 or the image pickup element 103. Thus, photographed image data ofthe object in the strobe photographing is obtained. Then, the processproceeds to Step S2.

In Step S2, the lens control unit 107 detects a current zoom position ofthe photographing lens 102 and a current focus position (focus lensposition) of the photographing lens 102. Then, the correction gaincalculation unit 113 a of the correction unit 113 obtains zoom positioninformation regarding the current zoom position of the photographinglens 102 and focus position information (focus lens positioninformation) regarding the current focus position of the photographinglens 102 from the lens control unit 107.

In Step S3, the exposure control unit 108 obtains, from the iris controlunit 109, iris value information regarding a current iris value of theiris 101 detected by the iris control unit 109. The correction gaincalculation unit 113 a obtains the iris value information regarding thecurrent iris value of the iris 101 from the exposure control unit 108.

In Step S4, the correction gain calculation unit 113 a obtains objectdistance distribution information regarding the object distancedistribution measured by the pupil-divided image phase differencedistance measurement unit 106. The object distance distributioninformation obtained by the pupil-divided image phase differencedistance measurement unit 106 is described below.

In Step S5, the correction gain calculation unit 113 a calculates animage correction gain on the basis of different types of informationobtained in Steps S2 to S4 and the strobe light distributioncharacteristic data stored in the strobe light distributioncharacteristic storing unit 112. The strobe light distributioncharacteristic data and the method of calculating the image correctiongain by the correction gain calculation unit 113 a are described belowin detail.

In Step S6, the correction processing unit 113 b of the correction unit113 performs processing of correcting the photographed image dataobtained by the strobe photographing of Step S1 by using the imagecorrection gain calculated by the correction gain calculation unit 113a. The correction processing of the correction processing unit 113 b isdescribed below in detail.

Subsequently, the photographed image data corrected by the correctionunit 113 is compressed by the compression processing unit 115 to bestored in the image data storing unit 116. Then, the processing of theflowchart is finished.

Steps S7 and S8 illustrated in FIG. 2 enable insertion of determinationprocessing as to whether to proceed from Step S1 to Step S2 underconditions described later. Steps S7 to S9 illustrated in FIG. 2 aredescribed below in detail referring to FIG. 40.

Phase Difference AF by Distance Measurement Pixel Disposed in ImagePickup Element 103

Next, auto-focus (phase difference AF) of a phase difference systemaccording to this embodiment is described. First, referring to FIGS. 3Ato 3C and FIGS. 4A and 4B, a pixel array of the image pickup element 103that is a basis for phase difference AF is described.

FIGS. 3A to 3C are schematic diagrams each illustrating an example of abasic pixel array of a general image pickup element. FIGS. 4A and 4B areschematic diagrams each illustrating an example of a basic pixel arrayof the image pickup element 103 according to the embodiment of thepresent invention.

First, the basic pixel array of the general image pickup elementillustrated in FIGS. 3A to 3C is described. Each of FIGS. 3A to 3Cillustrates a color array included in a basic unit portion of an areasensor where a basic unit is 2 pixels×2 pixels. FIG. 3A illustrates aso-called Bayer array, which is referred to as a pure color Bayer arraybelow. FIG. 3B illustrates an example where the Bayer array is appliedto a complementary color filter, which is referred to as a complementarycolor Bayer array below. FIG. 3C illustrates three complementary colorfilters with G, which is referred to as a complementary color array withG below.

As a generally known basic pixel array, in addition to the basic pixelarrays illustrated in FIGS. 3A to 3C, for example, there is acomplementary color checkered array of a unit of 2 pixels×4 pixels,which is often used as an image pickup element for a video movie camera.As another generally known basic pixel array, for example, there is acomplementary color checkered array of 2 pixels×8 pixels (refer toJapanese Patent Application Laid-Open No. H09-46715). The color pixelarrays of 2 pixels×4 pixels and 2 pixels×8 pixels are more advantageousas area sensors for processing moving images (videos performinginterlace scanning). As a camera for processing still images, the arrayof 2 pixels×2 pixels is more advantageous in that signal processing canbe simplified and a high-quality image can be obtained. The area sensorwith 2 pixels×2 pixels set as a basic unit is described below. However,this arrangement can be applied to the area sensors having color pixelarrays of 2 pixels×4 pixels and 2 pixels×8 pixels.

Next, referring to FIGS. 4A and 4B, the basic pixel array of the imagepickup element 103 of this embodiment is described. FIG. 4A illustratesa pixel array in the case of a pure color Bayer array, while FIG. 4Billustrates a pixel array in the case of a complementary color Bayerarray or a complementary color array with G.

In FIGS. 4A and 4B, a portion “S” is a function sensor cell (distancemeasurement pixel) for reading photometry data regarding distancemeasurement for AF. In the image pickup element 103 of this embodiment,a distance measurement pixel equivalent to an AF sensor is included inthe image pickup element 103 itself, and distance measurement for AF ofthe digital camera is carried out by reading a signal from the imagepickup element 103. Through this arrangement, this embodiment enableshighly accurate AF. There is no need to dispose any other AF sensors,and hence a compact and low-cost digital camera can be provided.

Next, a pixel (distance measurement pixel) to detect photometry data forAF distance measurement and the image pickup element 103 including thedistance measurement pixel according to this embodiment is described.

As an image pickup element for a digital still camera of a large numberof pixels, mainly, an interline-type CCD or a full-frame type CCD isused. The interline-type CCD is often used for a low-cost camera equalto or less than a ⅔-inch optical system, while the full-frame type CCDis often used for a high-cost camera equal to or more than a 1-inchoptical system. A biggest difference between these types is that signalcharges can be read even if a light enters the image pickup element inthe case of the interline-type CCD while no signal charge can not beread unless a mechanical shutter disposed in the front of the imagepickup element is closed in the case of the full-frame type CCD.

The inventors of the present invention have offered a solution byproposing a improved full-frame type CCD of a structure which includes astorage unit for storing charges of a few lines between an image area ofthe full-frame type CCD and a horizontal CCD. The inventors of thepresent invention have offered a partial reading driving method for AFin an opened state of the mechanical shutter based on the improvedfull-frame type CCD. Additionally, the inventors of the presentinvention have offered a method of partially reading only a part of theimage area necessary for AF, at high speed (high-speed clearing methodfor signal charges other than necessary portion) in the case of theinterline-type CCD.

Thus, both in the interline-type CCD and in the full-frame type CCD(improved type), signal charges of an area including the distancemeasurement pixel set in the image area can be read withoutopening/closing the mechanical shutter many times within a short periodof time. An example of the embodiment using the improved full-frame typeCCD is described below. However, the example can be applied to theinterline-type CCD.

Before description of the distance measurement pixel, the image sensor(color pixel) of the image pickup element 103 is described.

FIGS. 5A and 5B are schematic diagrams each illustrating an example of apixel structure of the image sensor (color pixel) of the image pickupelement 103 according to the embodiment of the present invention. FIG.5A illustrates a structure of the pixel of the image sensor of the imagepickup element 103 when viewed from the top, while FIG. 5B illustrates apixel structure of a section cut on the line I-I′ of FIG. 5A and itspotential profile. Specifically, each of FIGS. 5A and 5B mainlyillustrates a photoelectric conversion unit in the pixel of the imagesensor.

In FIGS. 5A and 5B, a clock gate electrode 201 is made of, for example,light-transparent polysilicon. A semiconductor layer surface under theclock gate electrode 201 is a clock phase area. The clock phase area isdivided into two areas by ion implantation. One is a clock barrier area202, and the other area is a clock well area 203 formed by implantingions so as to set higher potential than in the clock barrier area 202.

A virtual gate 204 is for fixing channel potential by forming a P⁺ layerin the semiconductor layer surface, and this formed area is a virtualphase area. The virtual phase area is divided into two areas byimplanting N-type ions to a layer deeper than the P⁺ layer. One is avirtual barrier area 205, and the other is a virtual well area 206.

An insulation layer 207 is made of, for example, an oxide film, anddisposed between the electrode and the semiconductor layer. A channelstop area 208 is for isolating a channel of each VCCD.

Though not illustrated in FIGS. 5A and 5B, a function of preventing ablooming phenomenon where charges overflow into an adjacent pixel togenerate a false signal when a strong light enters, is added. Itsrepresentative method is a method where a horizontal overflow drain isdisposed. Specifically, a drain including an N⁺ layer is disposedadjacently to each VCCD and an overflow drain barrier is disposedbetween the overflow drain and a charge transfer channel. In otherwords, charges exceeding a height of the overflow drain barrier areswept away by the drain. The height of the overflow drain barrier isfixed by ion implantation. However, disposing an electrode (overflowdrain barrier electrode) on the overflow drain barrier enables changingof the height under control of a value of a voltage (VOD) applied to thedrain electrode.

For VCCD transfer, an arbitrary pulse is added to the clock gateelectrode 201, and potential of the clock phase area is moved up anddown with respect to potential of the virtual phase area, therebytransferring charges toward the horizontal CCD. A concept of the chargemovement is indicated by “→o” of FIG. 5B.

The pixel structure of the image sensor of the image pickup element 103has been described. A pixel structure of the storage unit describedabove is similar. However, in the area of the storage unit, the pixelupper part is aluminum light-shielded, eliminating the necessity ofpreventing blooming. Thus, an overflow drain is omitted. An H-CCD has avirtual phase structure, and a layout of a clock phase area and avirtual phase area is configured so that charges can be received fromthe VCCD and transferred horizontally.

FIGS. 6A and 6B are schematic diagrams each illustrating an example of apixel structure of the image sensor (color pixel) of the image pickupelement 103 according to the embodiment of the present invention.Specifically, FIGS. 6A and 6B illustrate the pixel structure of theimage sensor which includes a color filter in an upper part of the pixelstructure (mainly, photoelectric conversion unit) illustrated in FIGS.5A and 5B. In FIGS. 6A and 6B, components similar to those of FIGS. 5Aand 5B are denoted by similar reference numerals. FIG. 6A illustratesthe structure when the pixel of the image sensor of the image pickupelement 103 is seen from the top, and FIG. 6B illustrates the structureof the pixel of a section cut on the line 6B-6B of FIG. 6A.

A passivation layer 209 is a passivation layer of the semiconductorlayer surface formed in the pixel structure illustrated in FIGS. 5A and5B. A metal layer 210 is a metal layer (metal light-shielding layer)formed between the pixel structure illustrated FIGS. 5A and 5B and acolor filter 212 to prevent mixing of colors. The metal layer 210 mayinclude a black pigment layer made of the same material as that of thecolor filter 212. A smoothing layer 211 is for smoothing a surface ofthe color filter 212. The color filter 212 includes one of pure colorsor one of complementary colors. A passivation layer 213 is forprotecting the color filter 212.

Next, a pixel array in a case where a pixel (distance measurement pixel)for detecting AF distance measurement data is configured in the imagepickup element 103 including a full-frame type CCD is described.

FIG. 7 is a schematic diagram illustrating an example of a pixel arrayof the image pickup element 103 according to the embodiment of thepresent invention. In FIG. 7, a normal Bayer array sensor includes aline having a plurality of distance measurement pixels S1 and a linehaving a plurality of distance measurement pixels S2 arranged side byside.

Pixel structures of the distance measurement pixels S1 and S2illustrated in FIG. 7 are described. FIGS. 8A and 8B are schematicdiagrams each illustrating an example of the pixel structure of thedistance measurement pixel S1 illustrated in FIG. 7. FIG. 8A illustratesthe structure when the distance measurement pixel S1 is seen from thetop, and FIG. 8B illustrates the pixel structure of a section cut on theline III-III′ of FIG. 8A. In FIGS. 8A and 8B, components similar tothose of FIGS. 6A and 6B are denoted by similar reference numerals.

The distance measurement pixel S1 includes no color filter 212 disposedin the image sensor (color pixel) illustrated in FIGS. 6A and 6B but amicro lens 216 disposed in its uppermost part.

A light-shielding layer 214 is formed on the smoothing layer 211illustrated in FIGS. 8A and 8B. The light-shielding layer 214 includes asmoothing layer 215 formed to form the micro lens 216 on a plane. Thissmoothing layer 215 is made of the same material as that of thepassivation layer 213 of the image sensor illustrated in FIGS. 6A and6B, and formed through the same process as that of the passivation layer213. A characteristic configuration of the distance measurement pixel S1is that on the same plane of the color mixing prevention light-shieldingmetal layer 210 in the image sensor illustrated in FIGS. 6A and 6B, thelight-shielding layer 214 having an aperture unbalanced (eccentric) inone side (leftward) from a center of the photoelectric conversion areaof the pixel is configured.

FIGS. 9A and 9B are schematic diagrams each illustrating an example of apixel structure of the distance measurement pixel S2 illustrated in FIG.7. FIG. 9A illustrates the structure when the distance measurement pixelS2 is seen from the top, and FIG. 9B illustrates the pixel structure ofa section cut on the line IV-IV′ illustrated in FIG. 9A. In FIGS. 9A and9B, components similar to those of FIGS. 8A and 8B are denoted bysimilar reference numerals.

In the distance measurement pixel S2 illustrated in FIGS. 9A and 9B, alight-shielding layer 214 is formed so that it can be provided with anaperture in the same distance from a pixel center in a directionopposite to that of the distance measurement pixel S1 illustrated inFIGS. 8A and 8B.

In the image pickup element 103 including more than 1 million pixels, inthe pixel array of FIG. 7, proximate optical images are formed on themicro lens 206 so that the row groups respectively including thedistance measurement pixels S1 and S2 are considered as almost the sameline. As long as the photographing lens 102 for forming an optical imagein the image pickup element 103 is in-focus on the image pickup element103, an image signal (pixel signal) from the row group including thedistance measurement pixel S1 and an image signal (pixel signal) fromthe row group including the distance measurement pixel S2 match eachother. If an image forming point connecting a focus of the photographinglens 102 is before or after the image area of the image pickup element103, a phase difference is generated between the image signal from therow group including the distance measurement pixel S1 and the imagesignal from the row group including the distance measurement pixel S2.In this case, deviation directions of phases are opposite between whenthe image forming point is before the image area and when it is afterthe image area. In principle, this phase difference is similar to thepupil-divided image phase difference AF described in Japanese PatentApplication Laid-Open No. H09-43507. In other words, when thephotographing lens 102 is viewed from the photoelectric conversion unitof the distance measurement pixel S1 and when the photographing lens 102is viewed from the photoelectric conversion unit of the distancemeasurement pixel S2, a pupil looks divided into left and right aroundan optical center.

FIGS. 10A and 10B are conceptual diagrams of detection of imagedeviation caused by an out-of-focus state of the image pickup element103 according to the embodiment of the present invention. In FIGS. 10Aand 10B, the distance measurement pixels S1 and S2 match each other, andare indicated by points A and B. For easier understanding, the colorpixel (image sensor) is omitted.

A light from a specific point of the object is divided into a light flux(ΦLa) passed through a pupil of A to enter the point A and a light flux(ΦLb) passed through a pupil of B to enter the point B. These two lightfluxes are originally emitted from one point (specific point of theobject). Thus, as long as an in-focus position of the photographing lens102 is on the image pickup element 103, as illustrated in FIG. 10A, thelight fluxes reach one point in the same micro lens 216. However, asillustrated in FIG. 10B, if an in-focus position of the photographinglens 102 is before the image pickup element 103 by a distance x, phasesdeviate from each other by 2θx. In this case, a reaching point deviatesin an opposite direction with the distance x.

According to this principle, an image (signal line based on lightintensity) formed by an A array and an image formed by a B array matcheach other as long as the photographing lens 102 is in-focus, and phasesdeviate from each other if not in-focus.

Based on this principle, the image pickup element 103 of this embodimentis formed by incorporating distance measurement pixels having microlenses 216 of different aperture positions in the basic pixel array.Specifically, the image pickup element 103 is formed by disposing anarea for adjacently arranging a row group of the basic pixel arrayincluding the distance measurement pixel S1 having a first aperture anda row group of the basic pixel array including the distance measurementpixel S2 having a second aperture. According to this embodiment, a phasedeviation amount between image signals from the row groups respectivelyincluding the distance measurement pixels S1 and S2 of this area iscalculated for detection to obtain an out-of-focus amount of thephotographing lens 102. Moving a focus position of the digital camera100 by the obtained out-of-focus amount enables auto focusing.

FIGS. 11, 12 and 13 are schematic diagrams each illustrating a modifiedexample of a pixel array of the image pickup element 103 according tothe embodiment of the present invention. FIGS. 11 to 13 illustrate waysof arranging the distance measurement pixels S1 and S2 changed from thatof the pixel array illustrated in FIG. 7.

In the aforementioned example, the row group including the distancemeasurement pixel S1 for detecting a first phase and the row groupincluding the distance measurement pixel S2 for detecting a second phaseslightly deviate from each other. As described above, this arrangementposes no practical problems in the image pickup element 103 includingmore than 1 million pixels. In the modified examples of FIGS. 11 to 13,the row groups for detecting the phases are brought closer to the sameplace.

Specifically, FIG. 11 illustrates an alternate arrangement of thedistance measurement pixels S1 and S2 on the same line. FIG. 12illustrates an arrangement of row groups including the distancemeasurement pixels S1 above and below the row group including thedistance measurement pixel S2 . In this case, data of the row groupincluding the distance measurement pixel S1 equivalent to the row groupincluding the distance measurement pixel S2 is obtained by interpolationbetween the row groups respectively including the distance measurementpixel S1 and the distance measurement pixel S2 . FIG. 13 illustrates amodified example of FIG. 11, where two row groups including the distancemeasurement pixels S1 and S2 on the same line are disposed adjacently toeach other, and positions of the distance measurement pixels S1 and S2are opposite to each other between the two row groups.

Thus, the driving method of reading the pixel group (row group includingthe distance measurement pixel) for generating a phase difference signaland only a part thereof enables high-speed and highly accurate AF.

In image processing for RAW data (CCD-ROW data; direct information ofeach pixel) regarding an image photographed in the image pickup element103, the distance measurement pixels S1 and S2 are processed to obtainan image by interpolation from the surrounding color pixels. Thus, theimage pickup element 103 can be realized, which can not only fetch aphotographed image but also read distance measurement data for AF withalmost no deterioration of image quality of the photographed image. Onthe presumption that such interpolation processing is executed, in thecase of a basic pixel array of 2 pixels×2 pixels similar to that ofFIGS. 4A and 4B, with three color pixels, and one distance measurementpixel, interpolation processing is simple, and image deterioration islittle troublesome. Needless to say, this arrangement is applicable evenin the case of a basic pixel array of 2 pixels×4 pixels. In this case, adistance between the row group including the distance measurement pixelS1 and the row group including the distance measurement pixel S2 islarger than that of 2 pixels×2 pixels.

The case where the image pickup element 103 is the improved full-frametype CCD has been described. The same applies even when the image pickupelement 103 is an interline-type CCD, a frame transfer type CCD, or anX-Y address type image pickup element.

Next, further improvable points of this embodiment are described.

FIGS. 14 and 15 are conceptual diagrams of focus detection of thedigital camera (image pickup apparatus) according to the embodiment ofthe present invention. For easier understanding, the distancemeasurement pixels S1 and S2 are illustrated on the same plane.

A light from a specific point of the object is divided into a light flux(L1) passing through a pupil of the distance measurement pixel S1 toenter the distance measurement pixel S1 and a light flux (L2) passingthrough a pupil of the distance measurement pixel S2 to enter thedistance measurement pixel S2 . When the photographing lens 102 of thedigital camera 100 is in-focus, these two light fluxes are converged onone point on the surface of the micro lens 216 as illustrated in FIG.14. In this case, the same optical image is exposed in the distancemeasurement pixels S1 and S2 . Thus, an image signal read from the rowgroup including the distance measurement pixel S1 and an image signalread from the row group including the distance measurement pixel S2 areidentical.

On the other hand, when the photographing lens 102 of the digital camera100 is out-of-focus, as illustrated in FIG. 15, the light fluxes L1 andL2 intersect each other at a position different from the surface of themicro lens 216. It is presumed here that a distance between the surfaceof the micro lens 216 and a intersection point of the two light fluxes,in other words, a defocus amount, is x, and a deviation amount betweenan image of the distance measurement pixel S1 and an image of thedistance measurement pixel S2 occurring at this time is n. Asillustrated in FIG. 15, a sensor pitch is d, a distance between centersof gravity of two pupils is Daf, and a distance from a principal pointof the photographing lens 102 to a focus is u.

In this case, the defocus amount x is calculated by the followingequation (1):x=n×d×u/Daf  (1)

The distance u is expected to be almost equal to a focal length f of thephotographing lens 102. The equation (1) is accordingly represented bythe following equation (2):x=n×d×f/Daf  (2)

FIG. 16 is a schematic diagram illustrating a status of image signalsread from the row groups respectively including the distance measurementpixels S1 and S2 of the image pickup element 103 according to theembodiment of the present invention. FIG. 16 illustrates, for example,the case illustrated in FIG. 15.

Image deviation n×d occurs between the image signals read from the rowgroups respectively including the distance measurement pixels S1 and S2. A deviation amount is obtained between these two image signals,thereby obtaining a defocus amount x by the equation (2). Moving thephotographing lens 102 by a distance x based on the obtained defocusamount x enables achievement of auto focusing.

To detect such image deviation, in the light that enters thephotographing lens 102, the light fluxes L1 and L2 passing through twodifferent pupils have to be separated from each other. According to thisembodiment, pupil separation is carried out by forming distancemeasurement pixels S1 and S2 having pupil separation functions in theimage pickup element 103.

FIGS. 17A and 17B are schematic diagrams each illustrating an example ofa pixel structure of the distance measurement pixel of the image pickupelement 103 according to the embodiment of the present invention. FIG.17A illustrates the structure when the distance measurement pixel isseen from the top, and FIG. 17B illustrates the pixel structure of asection cut on the line 17B-17B of FIG. 17A. The pixel structure of thedistance measurement pixel illustrated in FIGS. 17A and 17B is, forexample, similar to that illustrated in FIGS. 9A and 9B.

FIG. 17B illustrates the micro lens 216, the light-shielding layer 214,and an insulation layer 207 of a photoelectric conversion element. Alight enters the micro lens 216 from the photographing lens 102. A lightthat enters the photoelectric conversion element is only a light fluxentering from a specific direction limited by the light-shielding layer214.

FIGS. 18A and 18B are schematic diagrams illustrating thelight-shielding layers 214 in the distance measurement pixels S1 and S2of the image pickup element 103 according to the embodiment of thepresent invention. FIG. 18A illustrates the light-shielding layer 214 ofthe distance measurement pixel S1 as illustrated in FIGS. 8A and 8B.FIG. 18B illustrates the light-shielding layer 214 of the distancemeasurement pixel S2 as illustrated in FIGS. 9A and 9B.

As illustrated in FIGS. 18A and 18B, the light-shielding layers 214 ofthe distance measurement pixels S1 and S2 are disposed so that aperturesthereof can be left-right symmetrical. In this case, the apertures maybe up-down symmetrical. Thus, light fluxes from two pupil positionssymmetrical around an optical axis subject one of images formed on theimage pickup element 103 to photoelectric conversion by the row groupincluding the distance measurement pixel S1, and the other tophotoelectric conversion by the row group including the distancemeasurement pixel S2 . This way, two images of different pupil positionscan be obtained.

FIG. 19 is a schematic diagram illustrating an example of a pixel arrayof the image pickup element 103 according to the embodiment of thepresent invention. FIG. 19 illustrates the same pixel array as that ofFIG. 7.

A color pixel of the image pickup element 103 illustrated in FIG. 19includes a color filter of R, G or B. In FIG. 19, R indicates a colorpixel having a red color filter, G indicates a color pixel having agreen color filter, and B indicates a color pixel having a blue colorfilter. The distance measurement pixels S1 and S2 include no colorfilters but the light-shielding layers 214 and the micro lenses 216 asillustrated in FIG. 17B.

To perform an auto focus operation, the row groups including thedistance measurement pixels S1 and S2 are read from the image pickupelement 103, thereby subjecting a signal output from the image pickupelement 103 to A/D conversion at the A/D conversion unit 104. Then, forexample, the pupil-divided image phase difference distance measurementunit 106 generates images of the distance measurement pixels S1 and S2based on obtained pixel values, and calculates a correlation between thetwo images to obtain an image deviation amount. The lens control unit107 moves the photographing lens 102 according to the image deviationamount obtained by the pupil-divided image phase difference distancemeasurement unit 106 to achieve an auto focus operation.

To perform photographing, first, an object image is exposed on the imagepickup element 103 to read a signal from all the pixels of the imagepickup element 103. The image signal read from the image pickup element103 is subjected to A/D conversion at the A/D conversion unit 104 toenter the development processing unit 105. The development processingunit 105 discards pixel values read from the distance measurement pixelsS1 and S2 , and generates pixel values corresponding to the distancemeasurement pixels S1 and S2 from surrounding pixels instead, to therebyperform interpolation processing. The development processing unit 105performs development processing for image data by generating a luminanceand color difference signal. Then, the compression processing unit 115compresses the image data to store the compressed image data in theimage data storing unit 116.

As described above, according to the embodiment of the presentinvention, the distance measurement pixels S1 and S2 include no colorfilter. Thus, no photographed image data can be used, but a pixel valueof this portion is subjected to interpolation by using the surroundingpixels.

Interpolation processing during static image photographing may becarried out after the image data based on the image signal is fetchedinto the memory (e.g., image data storing unit 116). Interpolationprocessing during moving image photographing or in an electronicviewfinder is carried out by thinning out lines on the image pickupelement 103 to prevent processing delay because about 30 images arerepeatedly read from the image pickup element 103 per second during suchphotographing.

Thus, reading of the lines including the distance measurement pixels S1and S2 is completed. In the digital camera 100, for example, thepupil-divided image phase difference distance measurement unit 106selects two specific lines from among the read lines, and calculates adefocus amount from a phase difference between the distance measurementpixels S1 and S2 . The lens control unit 107 drives, based on thedefocus amount obtained by the pupil-divided image phase differencedistance measurement unit 106, the photographing lens 102 to completethe auto focus operation.

Thus, according to the embodiment of the present invention, an imagesignal of a pixel read during reading in the thinning mode includes nopixel signal of a distance measurement pixel, and a pixel signalincluding a number of pixels enough to generate a moving image can beread.

For example, dividing the pupil orthogonally to horizontal and verticaldirections of the image pickup element 103 enables improvement ofin-focus accuracy by detecting phase differences in both horizontal andvertical directions. Referring to FIGS. 20A and 20B, this arrangement isdescribed below.

FIGS. 20A and 20B are schematic diagrams each illustrating an example ofa distance measurement pixel of the image pickup element 103 accordingto the embodiment of the present invention. FIG. 20A illustrates adistance measurement pixel (S) pupil-divided in the horizontal directionof the image pickup element 103. FIG. 20B illustrates a distancemeasurement pixel (S) pupil-divided in the vertical direction of theimage pickup element 103.

Distance measurement pixels similar to those illustrated in FIGS. 20Aand 20B are disposed at constant intervals all over in the image pickupelement 103. Then, for example, about 1% of all the pixels of the imagepickup element 103 is allocated to the distance measurement pixels.Vertical stripes of the object photographed in a normal position of thedigital camera 100 are for detecting a phase difference (defocus amount)by the distance measurement pixel pupil-divided in the horizontaldirection (lateral direction). Horizontal stripes of the objectphotographed in the normal position of the digital camera 100 are fordetecting a phase difference (defocus amount) by the distancemeasurement pixel pupil-divided in the vertical direction (longitudinaldirection). Focusing by using those phase differences (defocus amounts)enables highly accurate setting of an in-focus state.

Object distance distribution of an image photographed within an angle ofview is obtained for each area by such a distance measurement system asdescribed above. This is described below referring to the drawings.

FIG. 21 is a schematic diagram illustrating a status of strobephotographing of an object using the digital camera (image pickupapparatus) of the embodiment of the present invention.

A wall 2101 includes a wall-hung clock 2102. A first person 2103 standsbefore the wall 2101. A second person 2104 stands before the firstperson 2103 with respect to the wall 2101. In FIG. 21, body parts otherthan heads of the first and second persons 2103 and 2104 are simplifiedby straight lines. Photographing ranges 2105 and 2106 indicate angles ofview. FIG. 21 illustrates only the photographing lens 102 and the imagepickup element 103 of the digital camera 100. In the image pickupelement 103, an object image formed through the photographing lens 102is illustrated.

FIG. 22 is a schematic diagram illustrating an example of a photographedimage when strobe photographing of the object illustrated in FIG. 21 iscarried out. FIG. 22 illustrates an image photographed by the imagepickup element 103 as an example. In FIG. 22, as in the case of FIG. 21,body parts other than the head of the first person 2103 are simplifiedby the straight lines.

FIG. 23 is a schematic diagram illustrating an example of objectdistance distribution in the strobe photographing of the objectillustrated in FIG. 21. An angle of view 2301 indicates an angle of viewof the photographed image. The object distance distribution of FIG. 23is obtained by the pupil-divided image phase difference distancemeasurement unit 106. In FIGS. 21 and 22, the body parts other than thehead of the first person 2103 are illustrated in the simplified manner.In FIG. 23, however, the body parts are illustrated not in a simplifiedmanner.

In FIG. 23, a distance 2302 indicates a distance to the wall 2101. Inthe example of FIG. 23, a distance from the digital camera 100 is 303cm. A distance 2303 indicates a distance to the wall-hung clock 2102. Inthe example of FIG. 23, a distance from the digital camera 100 is 300cm.

Distances 2304 to 2308 indicate distances to the first person 2103. Inthe example of FIG. 23, distances from the digital camera 100 to thefirst person 2103 have distance distribution of 200 cm to 206 cm.Distances 2309 to 2311 indicate distances to the second person 2104. Inthe example of FIG. 23, distances from the digital camera 100 to thesecond person 2104 have distance distribution of 100 cm to 102 cm. FIG.23 illustrates an x axis 2312 indicating a horizontal positioncoordinate of the photographed image, and a y axis 2313 indicating avertical position coordinate of the photographed image. Object distancedistribution D of the object can be represented by D(x, y).

Description of Strobe Light Distribution Characteristics

Next, strobe light distribution characteristics of the embodiment of thepresent invention are described.

FIGS. 24A to 24E are schematic diagrams illustrating strobe lightdistribution characteristics.

FIG. 24A is a perspective diagram schematically illustrating arelationship for far and near objects between light distribution of thestrobe 111 and each of the photographing lens 102 and the image pickupelement 103. FIG. 24B illustrates horizontal light distributioncharacteristics D1 of a flash of the strobe 111, for the near object.FIG. 24C illustrates vertical light distribution characteristics D2 ofthe flash of the strobe 111, for the near object. FIG. 24D illustrateshorizontal light distribution characteristics D3 of a flash of thestrobe 111, for the far object. FIG. 24E illustrates vertical lightdistribution characteristics D4 of the flash of the strobe 111, for thefar object.

Specifically, FIGS. 24B and 24D illustrate light distributioncharacteristics in a range from a vertical surface including anintersection point between a center axis 27 of light distribution of thestrobe 111 illustrated in FIG. 24A and a near object surface 28 a or afar object surface 28 b, to a left-and-right direction. FIGS. 24C and24E illustrate light distribution characteristics in a range from ahorizontal surface including the intersection point between the centeraxis 27 of the light distribution of the strobe 111 illustrated in FIG.24A and the near object surface 28 a or the far object surface 28 b, toan up-and-down direction.

Thus, the vertical surface including the intersection point between thecenter axis 27 of light distribution and the object surface illustratedin FIGS. 24B and 24D of light distribution characteristics based on aflash hat 111 a matches a lens axis 29 of the photographing lens 102(when the strobe 111 is directly above the photographing lens 102).

However, the horizontal surface including the intersection point betweenthe center axis 27 of light distribution and the object surfaceillustrated in FIGS. 24C and 24E deviates upward from the near objectsurface 28 a and downward from the far object surface 28 b asillustrated in FIG. 24A.

In this case, it is presumed that arrangement of the strobe 111 is laidout so that an optical axis of light distribution of the strobe 111 andan optical axis of the photographing lens 102 can match each other in ahorizontal direction (camera left-and-right direction) and deviate fromeach other in a vertical direction (camera up-and-down direction). It isalso presumed that the optical axis of light distribution of the strobe111 is tilted so that the optical axis of light distribution of thestrobe 111 and the optical axis of the photographing lens 102 canintersect each other in the vertical direction at a predetermineddistance. In other words, with respect to the lens axis 29 of thephotographing lens 102, i.e., a center of the image pickup element 103,the light distribution characteristics D2 move upward as illustrated inFIG. 24C in the case of the near object, and the light distributioncharacteristics D4 move downward as illustrated in FIG. 24E in the caseof the far object. This processing enables obtaining of highly accuratestrobe light distribution characteristics which take into considerationparallax caused by a positional difference between the photographinglens 102 as a camera main lens and the strobe 111.

FIGS. 25A and 25B are schematic diagrams each illustrating an example ofstrobe light distribution characteristics based on a zoom position. FIG.25A illustrates horizontal strobe light distribution characteristicsbased on the zoom position. FIG. 25B illustrates vertical strobe lightdistribution characteristics based on the zoom position.

In FIG. 25A, an ordinate indicates a light amount of light distributionof the strobe 111, while an abscissa indicates a light spread rangearound the center axis 27 of light distribution of the strobe 111. Lightdistribution characteristics H illustrated in FIG. 25A show a degree oflight amount attenuation around a horizontal direction with a lightamount a of the center axis 27 of light distribution being set to 100%.

In the degree of light amount attenuation of the horizontal directionillustrated in FIG. 25A, as indicated by the light distributioncharacteristics H, during photographing at a wide end, a light amount isattenuated along a curve from a center position of the light amount a toh1 in a left direction and to h4 in a right direction as illustrated inFIG. 25A. This attenuated light amount is reflected on brightness of anobject image formed on an image pickup element 103 w. Duringphotographing at a tele end, a light amount is attenuated along a curvefrom the center position of the light amount a to h2 in the leftdirection and to h3 in the right direction as illustrated in FIG. 25A.This attenuated light amount is reflected on brightness of an objectimage formed on an image pickup element 103 t.

In FIG. 25B, an abscissa indicates a light amount of light distributionof the strobe 111, while an ordinate indicates a light spread rangearound the center axis 27 of light distribution of the strobe 111. Lightdistribution characteristics P illustrated in FIG. 25B show a degree oflight amount attenuation around a vertical direction with a light amounta of the center axis 27 of light distribution being set to 100%. Now,consideration is given to a case where the optical axis of thephotographing lens 102 matches the centers of the image pickup element103 and the object surface. In this case, when a center light amount ofan object image formed on the image pickup element 103 is 100%, lightamounts received by individual pixels arrayed from the center of theobject image in the vertical direction on the image pickup element 103can be calculated.

In the degree of light amount attenuation of the vertical directionillustrated in FIG. 25B, as indicated by the light distributioncharacteristics P, during photographing at the wide end, a light amountis attenuated along a curve from the center position of the light amounta to p1 in an upper direction and to p4 in a lower direction asillustrated in FIG. 25B. This attenuated light amount is reflected onbrightness of an object image formed on the image pickup element 103 w.During photographing at the tele end, a light amount is attenuated alonga curve from the center position of the light amount a to p2 in theupper direction and to p3 in the lower direction as illustrated in FIG.25B. This attenuated light amount is reflected on brightness of anobject image formed on the image pickup element 103 t.

An attenuation amount of a light amount of a middle portion between thehorizontal and vertical directions is obtained as data of a curveconnecting the light distribution characteristics H and P. FIGS. 25A and25B illustrate the degrees of light amount attenuation in associationwith the spread ranges of light distribution characteristics, and thussizes of the image pickup elements 103 w and 103 t are different fromeach other. Actually, however, the image pickup elements 103 w and 103 tindicate the same image pickup element 103.

The light distribution characteristics of the strobe 111 based on theobject distance or the zoom position as described above are arrangedinto a table as correction data (strobe light distributioncharacteristic data) corresponding to a position of each pixel of theimage pickup element 103, and stored beforehand in, for example, thestrobe light distribution characteristic storing unit 112. Thus,according to the embodiment of the present invention, the strobe lightdistribution characteristic data stored in the strobe light distributioncharacteristic storing unit 112 contains a zoom position of thephotographing lens 102. The strobe light distribution characteristicstoring unit 112 is, for example, a nonvolatile memory such as a flashmemory, and the strobe light distribution characteristic data is storedbeforehand in this predetermined area.

The strobe light distribution characteristic data is represented bySY(D, Z, x, y), where D is an object distance, Z is a zoom position ofthe photographing lens 102, x is a horizontal coordinate, and y is avertical coordinate. In this case, in view of a memory capacity of thestrobe light distribution characteristic storing unit 112, whenarrangement of strobe light distribution characteristic data SY of allD, Z, x and y into a table is difficult, for example, the following canbe implemented. Specifically, a method may be employed, which storesstrobe light distribution characteristic data SY of discrete D, Z, x andy in a table, and obtains actual strobe light distributioncharacteristics SY from a nearest value among discrete D, Z, x and y byinterpolation processing. For the interpolation processing in this case,a bilinear method or a bicubic method may be used.

Description of Interpolation Processing

Next, interpolation processing carried out by the correction unit 113 isdescribed.

First, interpolation processing when an in-focus state is not taken intoconsideration is described.

FIG. 26 is a schematic diagram illustrating an example of strobe lightdistribution characteristics.

In FIG. 26, strobe light distribution characteristics 2601 and 2602indicate fixed-direction strobe light distribution characteristics SY ofa certain object distance D, a zoom position Z, and a coordinate (x, y).In other words, light distribution of the strobe 111 is bright at thecenter of an angle of view, and is dark at the periphery because ofinsufficient illumination.

FIG. 27 is a schematic diagram illustrating an example of a correctiongain in the case of the strobe light distribution characteristicsillustrated in FIG. 26. In FIG. 27, a correction gain 2601′ is for thestrobe light distribution characteristics 2601 of FIG. 26, and acorrection gain 2602′ is for the strobe light distributioncharacteristics 2602 of FIG. 26. A correction gain for correction toappropriate brightness can be represented by G(D, Z, x, y). In thiscase, for example, the correction gain G(D, Z, x, y) can be defined asan inverse number of the strobe light distribution characteristics SY(D,Z, x, y). Only the correction gain of one direction is illustrated here.However, actual correction processing is carried out for atwo-dimensional image area by the correction gain G(D, Z, x, y).

Next, correction processing when object distance distribution in thephotographed image is taken into consideration is described.

For example, when photographing accompanied by light emission of thestrobe 111 is carried out in a picture composition illustrated in FIG.21, an image similar to that illustrated in FIG. 22 is photographed. Inthis case, the second person 2104 near the digital camera 100 is bright,the first person 2103 away from the digital camera 100 is darker thanthe second person 2104, and the wall 2101 and the clock 2102 more awaytherefrom are even darker. The object becomes darker from the center tothe peripheral portion thereof.

In the case of such photographing as described above, an object distanceD varies from one coordinate (x, y) to another in the photographedimage. In this case, object distance distribution D(x, y) obtained fromthe pupil-divided image phase difference distance measurement unit 106becomes as illustrated in FIG. 23 as described above. The correctiongain calculation unit 113 a obtains, using the object distancedistribution D(x, y), strobe light distribution characteristics SY(D(x,y), Z, x, y) for each object distance from the strobe light distributioncharacteristics SY(D, Z, x, y) stored in the strobe light distributioncharacteristic storing unit 112. Then, the correction gain calculationunit 113 a calculates a correction gain G(D(x, y), Z, x, y) based on theobtained strobe light distribution characteristics.

FIG. 28 is a schematic diagram illustrating an example of a photographedimage when the object illustrated in FIG. 21 is photographed. FIG. 28 isbased on the photographed image illustrated in FIG. 22, and illustratesa horizontal image portion 2801 of the photographed image.

FIG. 29 is a schematic diagram illustrating an example of objectdistance distribution of the horizontal image portion 2801 illustratedin FIG. 28. FIG. 29 illustrates object distance distribution D(x, y) foran x coordinate of an abscissa of the horizontal image portion 2801illustrated in FIG. 28.

In FIG. 29, object distance distributions 2901 to 2903 indicate distancedistributions of the wall 2101 from the digital camera 100. Objectdistance distribution 2904 indicates distance distribution of the firstperson 2103 from the digital camera 100, and object distancedistribution 2905 indicates distance distribution of the second person2104 from the digital camera 100.

FIG. 30 is a schematic diagram illustrating an example of a correctiongain in the horizontal image portion 2801 illustrated in FIG. 28. Thecorrection gain illustrated in FIG. 30 is a correction gain G(D(x, y),Z, x, y) obtained from the strobe light distribution characteristicsSY(D(x, y), Z, x, y) and the object distance distribution D(x, y)illustrated in FIG. 29.

Correction gains 3001 to 3003 indicate correction gains applied to thehorizontal image portion 2801 of the wall 2101. A correction gain 3004indicates a correction gain applied to the horizontal image portion 2801of the first person 2103, and a correction gain 3005 is a correctiongain applied to the horizontal image portion 2801 of the second person2104. Correction processing of the correction processing unit 113 b iscarried out by multiplying image data of the horizontal image portion2801 by each correction gain G(D(x, y), Z, x, y) illustrated in FIG. 30according to an x coordinate of the horizontal image portion 2801 of thephotographed image illustrated in FIG. 28. Similarly, in the correctionprocessing of the correction processing unit 113 b, a correction gainG(D(x, y), Z, x, y) of each pixel is obtained for a two-dimensionalphotographed image (x, y), and a corrected image (x, y) is obtained bymultiplying a corresponding photographed image by the correction gainG(D(x, y), Z, x, y).

FIG. 31 is a schematic diagram illustrating an example of a photographedimage after correction processing of the photographed image illustratedin FIG. 22.

As described above, performing correction processing by using thecorrection gain G based on the object distance distribution D enables,even in the case of an object having nonuniform distance distribution asillustrated in FIG. 23 with the picture composition illustrated in FIG.21, obtaining of a uniformly and appropriately bright photographed imagesimilar to that illustrated in FIG. 31.

Next, correction processing when an in-focus state is taken intoconsideration is described.

The correction gain calculation unit 113 a can obtain a degree ofin-focus FL(Z, F, A, D) from zoom position information Z and focusposition information F obtained from the lens control unit 107 and irisvalue information A obtained from the exposure control unit 108. In thiscase, a depth of object Df(Z, F, A, D1) determined to be in an in-focusstate at a certain degree of in-focus FL₀ or more is obtained. D denotesan object distance, and D1 denotes an in-focus object distance.

FIGS. 32 and 33 are schematic diagrams each illustrating an example of adegree of in-focus FL(Z, F, A, D). FIG. 32 illustrates an open state ofthe iris 101 as compared with the case of FIG. 33. In FIGS. 32 and 33,FL₀ 3202 and 3302 indicate degrees of in-focus at which determination ofan in-focus state is enabled, and D1 (3201, 3301) indicates an in-focusobject distance.

In this case, D2 (3203, 3303) indicates an in-focus limitation of afront side, D3 (3204, 3304) indicates an in-focus limitation of a rearside, and a range of from D2 (3203, 3303) to D3 (3204, 3304) indicates adepth of object Df to enable determination of an in-focus state. In thecase of FIG. 32, the depth of object Df is lower because the iris 101 isopen as compared with that of FIG. 33.

A degree of in-focus FL(Z, F, A, D(x, y)) can be obtained for acoordinate (x, y) of the photographed image from the degree of in-focusFL(Z, F, A, D) and the object distance distribution D(x, y) describedabove.

FIG. 34 is a schematic diagram illustrating an example of a degree ofin-focus FL(Z, F, A, D(x, y)) for the horizontal image portion 2801illustrated in FIG. 28. It is presumed here that the camera is focusedon the first person 2103 illustrated in FIG. 21.

Degrees of in-focus 3401 to 3403 indicate degrees of in-focus FL(Z, F,A, D(x, y)) of the wall 2101. A degree of in-focus 3404 indicates adegree of in-focus FL(Z, F, A, D(x, y)) of the first person 2103, and adegree of in-focus 3405 indicates a degree of in-focus FL(Z, F, A, D(x,y)) of the second person 2104.

Executing processing similar to the above enables obtaining of a degreeof in-focus FL(Z, F, A, D(x, y)) of the photographed image (x, y).Whether or not object distance distribution D(x, y) is within the depthof object Df to enable determination of an in-focus state can bedetermined. In other words, an in-focus state can be discriminated fromother states.

Correction processing with consideration given to object distancedistribution D(x, y) and a degree of in-focus FL(Z, F, A, D(x, y)) isdescribed.

FIG. 35 is a flowchart illustrating an example of a detailed processingprocedure in the image correction processing of Step S6 illustrated inFIG. 2.

First, in Step S61, the correction processing unit 113 b initializesvalues of x and y of a coordinate (x, y) to 0.

In Step S62, the correction processing unit 113 b determines whether ornot an object distance D(x, y) of the coordinate (x, y) is equal to ormore than a predetermined distance D₀. In other words, in Step S62, thecorrection processing unit 113 b determines whether or not the objectdistance D(x, y) of the coordinate (x, y) is equal to or less than apredetermined threshold value. If a result of the determination showsthat the object distance D(x, y) is not equal to or less than thedistance D₀, in other words, larger than the distance D₀, the processproceeds to Step S65 without executing correction processing for a pixelof the coordinate (x, y) of the photographed image.

On the other hand, if a result of the determination in Step S62illustrates that the object distance D(x, y) is equal to or less thanthe distance D₀, the process proceeds to Step S63. In Step S63, thecorrection processing unit 113 b determines whether or not a degree ofin-focus FL(Z, F, A, D(x, y)) of the coordinate (x, y) is equal to ormore than FL₀ to enable determination of an in-focus state. In otherwords, in Step S63, the correction processing unit 113 b determineswhether or not the degree of in-focus FL(Z, F, A, D(x, y)) of thecoordinate (x, y) is equal to or more than a predetermined thresholdvalue. If a result of the determination illustrates that the degree ofin-focus FL(Z, F, A, D(x, y)) of the coordinate (x, y) is not equal toor more than FL₀ (in other words, it is less than FL₀), determining thatan in-focus state has not been set, the correction processing unit 113 bproceeds to Step S65 without executing correction processing for thepixel of the coordinate (x, y) of the photographed image.

If a result of the determination in Step S63 illustrates that the degreeof in-focus FL(Z, F, A, D(x, y)) of the coordinate (x, y) is equal to ormore than FL₀, determining that an in-focus state has been set, thecorrection processing unit 113 b proceeds to Step S64. In Step S64, thecorrection processing unit 113 b carries out correction processing basedon the correction gain G(D(x, y), Z, x, y) obtained in Step S5 of FIG. 2for the pixel of the coordinate (x, y) of the photographed image.

In Step S65, the correction processing unit 113 b determines whether ornot the coordinate (x, y) is a last pixel of the photographed image. Ifa result of the determination illustrates that the coordinate (x, y) isnot a last pixel of the photographed image, the correction processingunit 113 b proceeds to Step S66. In Step S66, the correction processingunit 113 b sets values of x and y of a next coordinate (x, y) to returnto Step S62. Then, until processing of a last pixel of the photographedimage, the correction processing unit 113 b repeats Steps S62 to S66.

On the other hand, if a result of the determination in Step S65illustrates that the coordinate (x, y) is a last pixel of thephotographed image, the processing of the flowchart (image correctionprocessing illustrated in FIG. 2) is finished.

FIG. 36 is a schematic diagram illustrating an example of objectdistance distribution of the horizontal image portion 2801 illustratedin FIG. 28. FIG. 36 illustrates object distance distribution D(x, y) foran x axis of an abscissa of the horizontal image portion 2801illustrated in FIG. 28. In FIG. 36, components similar to those of FIG.29 are denoted by similar reference numerals.

In the object distance distribution D(x, y) illustrated in FIG. 36, forexample, when the above-mentioned distance D₀ is a distance 3601,distances of the wall 2101 are larger than the distance D₀ (distance3601) up to the object distance distributions 2901 to 2903. Thus,corresponding pixels are not targets of correction processing. On theother hand, the object distance distribution 2904 of the first person2103 and the object distance distribution 2905 of the second person 2104are equal to or less than the distance D₀ (distance 3601). Thus,corresponding pixels are targets of correction processing.

As illustrated in Step S62 of FIG. 35, by targeting only pixels in whichobject distance distribution D(x, y) is equal to or less than D₀ forcorrection, for example, only strobe light distribution to a nearestperson during photographing in a night view mode can be corrected, andbackground buildings can be set outside targets of correction.

FIG. 37 is a schematic diagram illustrating an example of a degree ofin-focus FL(Z, F, A, D(x, y)) of the horizontal image portion 2801illustrated in FIG. 28.

In FIG. 37, degrees of in-focus 3701 to 3703 indicate degrees ofin-focus FL(Z, F, A, D(x, y)) of the wall 2101 in the horizontal imageportion 2801. A degree of in-focus 3704 indicates a degree of in-focusFL(Z, F, A, D(x, y)) of the first person 2103 in the horizontal imageportion 2801, and a degree of in-focus 3705 indicates a degree ofin-focus FL(Z, F, A, D(x, y)) of the second person 2104 in thehorizontal image portion 2801. It is presumed here, for example, thatFL₀ to enable determination of an in-focus state is FL₀ (3706)illustrated in FIG. 37.

In this case, in Step S63, the degree of in-focus 3704 of the firstperson 2103 (similar for the degree of in-focus 3701 to 3703 of the wall2101) is less than FL₀(3706). Thus, a corresponding pixel is not atarget of correction processing. On the other hand, the degree ofin-focus 3705 of the second person 2104 is equal to or more thanFL₀(3706). Thus, a corresponding pixel is a target of correctionprocessing.

Executing the processing of Step S63 enables execution of correctionprocessing based on strobe light distribution characteristics only to anobject of an in-focus state. Through this processing, a blurred objectimage area of an out-of-focus state of the photographed image is setoutside a target of correction processing based on strobe lightdistribution characteristics, and only an object image area determinedto be in an in-focus state can be subjected to correction processing.

Next, a menu setting example regarding correction processing based onstrobe light distribution characteristics is described. FIGS. 38 and 39are schematic diagrams each illustrating an example of a menu settingscreen regarding correction processing based on strobe lightdistribution characteristics. This menu setting screen is, for example,displayed on a display screen of the operation input unit 114illustrated in FIG. 1 to receive an operation entry from an operator(user).

First, the menu setting screen of FIG. 38 is described.

FIG. 38 illustrates an example of a menu setting screen for settingwhether to carry out correction processing based on strobe lightdistribution characteristics for each photographing mode. In otherwords, the digital camera 100 of this embodiment is configured so thatthe operator can specify whether to correct the photographed image foreach photographing mode via the operation input unit 114.

When a button 3801 is selected, the correction unit 113 sets, for allthe photographing modes, nonexecution of correction processing (nocorrection) based on strobe light distribution characteristics. When thebutton 3801 is selected, selections by other buttons are cleared.

When a button 3802 is selected, the correction unit 113 sets, for allthe photographing modes, execution of correction processing (there iscorrection) based on strobe light distribution characteristics. When thebutton 3802 is selected, selections by other buttons are cleared.

Buttons 3803 to 3806 are for setting whether to carry out correctionprocessing based on strobe light distribution characteristics, for eachphotographing mode enabling photographing in the digital camera 100. Inthis case, various photographing modes selected by the buttons 3803 to3806 are separately set via the operation input unit 114 at, forexample, the time of actual photographing.

Specifically, when the button 3803 is selected, the correction unit 113sets execution of correction processing based on strobe lightdistribution characteristics when a photographing mode of the digitalcamera 100 is an auto photographing mode. When the button 3804 isselected, the correction unit 113 sets execution of correctionprocessing based on strobe light distribution characteristics when aphotographing mode of the digital camera 100 is a portrait photographingmode. When the button 3805 is selected, the correction unit 113 setsexecution of correction processing based on strobe light distributioncharacteristics when a photographing mode of the digital camera 100 is anight view photographing mode. When the button 3806 is selected, thecorrection unit 113 sets execution of correction processing based onstrobe light distribution characteristics when a photographing mode ofthe digital camera 100 is a macrophotographing mode. When one of thebuttons 3803 to 3806 is selected in a selected state of the button 3801or 3802, setting by the selected button 3801 or 3802 is cleared.

When a button 3807 is selected, the correction unit 113 cancels settingsselected by the respective buttons (3801 to 3806). When a button 3808 isselected, based on a currently selected state, the correction unit 113sets correction processing based on strobe light distributioncharacteristics. In this case, the correction unit 113 sets correctionprocessing based on strobe light distribution characteristics by, forexample, storing setting information set by the menu setting screen ofFIG. 38 in its own internal memory (e.g., internal memory of thecorrection gain calculation unit 113 a).

In the example of FIG. 38, the buttons 3804 and 3806 are selected.Accordingly, when settings are made, in the case where photographingmodes of the digital camera 100 are a portrait photographing mode and amacrophotographing mode, correction processing based on strobe lightdistribution characteristics is carried out.

The menu setting screen of FIG. 39 is described below.

FIG. 39 illustrates an example of a menu setting screen for settingwhether to carry out correction processing based on strobe lightdistribution characteristics for each type of a photographing lens 102.Setting by the menu setting screen of FIG. 39 is effective in the caseof an image pickup apparatus in which a photographing lens is detachableor in the case of an image pickup apparatus in which a conversion lensis detachable. In other words, the digital camera 100 of this embodimentis configured so that the operator can designate whether to correct thephotographed image for each type of the photographing lens 102 via theoperation input unit 114.

When a button 3901 is selected, the correction unit 113 setsnonexecution of correction processing (no correction) based on strobelight distribution characteristics for all photographing lens types.When the button 3901 is selected, selections by other buttons arecleared.

When a button 3902 is selected, the correction unit 113 sets, for allthe photographing lens types, execution of correction processing (thereis correction) based on strobe light distribution characteristics. Whenthe button 3902 is selected, selections by other buttons are cleared.

Buttons 3903 to 3906 are for setting whether to carry out correctionprocessing based on strobe light distribution characteristics, for eachtype of a photographing lens 102 loaded on the digital camera 100. Inthis case, various types of photographing lenses 102 selected by thebuttons 3903 to 3906 are separately set via the operation input unit 114at, for example, the time of loading the photographing lenses 102 on theimage pickup apparatus.

Specifically, when the button 3903 is selected, the correction unit 113sets execution of correction processing based on strobe lightdistribution characteristics when a type of a photographing lens 102 isa standard lens. When the button 3904 is selected, the correction unit113 sets execution of correction processing based on strobe lightdistribution characteristics when a type of a photographing lens 102 isa telescopic lens. When the button 3905 is selected, the correction unit113 sets execution of correction processing based on strobe lightdistribution characteristics when a type of a photographing lens 102 isa wide-angle lens. When the button 3906 is selected, the correction unit113 sets execution of correction processing based on strobe lightdistribution characteristics when a type of a photographing lens 102 isa macrolens. When one of the buttons 3903 to 3906 is selected in aselected state of the button 3901 or 3902, setting by the selectedbutton 3901 or 3902 is cleared.

When a button 3907 is selected, the correction unit 113 cancels settingsselected by the respective buttons (3901 to 3906). When a button 3908 isselected, based on a currently selected state, the correction unit 113sets correction processing based on strobe light distributioncharacteristics. In this case, the correction unit 113 sets correctionprocessing based on strobe light distribution characteristics by, forexample, storing setting information set by the menu setting screen ofFIG. 39 in its own internal memory (e.g., internal memory of thecorrection gain calculation unit 113 a).

In the example of FIG. 39, the buttons 3903 and 3905 are selected.Accordingly, when settings are made, in the case where types ofphotographing lenses 102 are a standard type and a wide-angle type,correction processing based on strobe light distribution characteristicsis carried out.

The photographing modes and the types of the photographing lenses 102illustrated in FIGS. 38 and 39 are only examples. Other photographingmodes and other types of photographing lenses 102 can be appliedsimilarly to this embodiment described above. The menu setting screensare also only examples, and settings by other similar menurepresentations may be employed.

FIG. 40 is a flowchart illustrating an example of proceeding processingto the flowchart of FIG. 2 when various settings are made by the menusetting screens illustrated in FIGS. 38 and 39.

When there are settings made by the menu setting screens of FIGS. 38 and39, after an end of Step S1 of FIG. 2, the process proceeds to Step S10illustrated in FIG. 40. In Step S10, the correction unit 113 determineswhether a current photographing mode of the digital camera 100 is aphotographing mode, which is set on the menu setting screen of FIG. 38,for executing correction processing based on strobe light distributioncharacteristics. In this case, for example, the correction unit 113makes the determination by reading setting information set on the menusetting screen of FIG. 38 and stored in its own internal memory.

If a result of the determination in Step S10 indicates that the currentphotographing mode of the digital camera 100 is not a photographing modeset in FIG. 38 to execute correction processing based on strobe lightdistribution characteristics, the correction unit 113 proceeds to StepS9 of FIG. 2 to finish the processing.

On the other hand, if a result of the determination in Step S10indicates that the current photographing mode of the digital camera 100is a photographing mode set in FIG. 38 to execute correction processingbased on strobe light distribution characteristics, the correction unit113 proceeds to Step S11.

In Step S11, the correction unit 113 determines whether a type of aphotographing lens 102 currently loaded on the digital camera 100 is aphotographing lens set in FIG. 39 to execute correction processing basedon strobe light distribution characteristics. In this case, for example,the correction unit 113 makes the determination by reading settinginformation set on the menu setting screen of FIG. 39 and stored in itsown internal memory.

If a result of the determination in Step S11 indicates that the type ofthe photographing lens 102 currently loaded on the digital camera 100 isnot a photographing lens set in FIG. 39 to execute correction processingbased on strobe light distribution characteristics, the correction unit113 proceeds to Step S9 of FIG. 2 to finish the processing.

On the other hand, if a result of the determination in Step S11indicates that the type of the photographing lens 102 currently loadedon the digital camera 100 is a photographing lens set in FIG. 39 toexecute correction processing based on strobe light distributioncharacteristics, the correction unit 113 proceeds to Step S9 of FIG. 2.Then, in this case, processing to proceed to Step S2 of FIG. 2 isexecuted to carry out correction processing based on strobe lightdistribution characteristics.

This embodiment has been described by way of the example of the imagepickup apparatus where the conversion lens can be loaded or thephotographing lens 102 can be detachably loaded. In the case of an imagepickup apparatus where a photographing lens 102 is fixed, settingregarding a type of a photographing lens illustrated in FIG. 39 is madeunnecessary. In this case, processing in Step S11 of FIG. 40 isunnecessary.

According to this embodiment, a distance from the object is measured foreach image area of the photographed image, and the measured distance isreflected to correct the photographed image. Thus, local radiationdistribution unevenness caused by a strobe can be removed from an imagephotographed by using the strobe. In other words, this embodimentenables adaptive brightness correction even for a nonlinear image areaof the photographed image.

The Steps of FIGS. 2, 35, and 40 illustrating the control method of thedigital camera (image pickup apparatus) 100 of this embodiment arerealized by executing a program stored in a RAM or a ROM via a CPU of acomputer. This program and a computer-readable recording mediumrecording the program are within the present invention.

Specifically, the program is, for example, stored in a recording mediumsuch as a CD-ROM, or supplied to the computer via various transmissionmedia. As the recording medium recording the program, in addition to theCD-ROM, a flexible disk, a hard disk, a magnetic tape, a magneto-opticaldisk, or a nonvolatile memory card can be used. As the transmissionmedium of the program, a communication medium in a computer network(LAN, WAN of the Internet, or radio communication network) system forpropagating program information as a carrier wave to supply the programcan be used. As a communication medium in this case, a wire line such asan optical fiber or a wireless line is available.

The present invention is not limited to the mode of realizing thefunctions of the digital camera (image pickup apparatus) 100 of thisembodiment by executing the supplied program via the computer. When theprogram cooperates with an operating system (OS) operating in thecomputer or other application software to realize the functions of thedigital camera (image pickup apparatus) 100 of this embodiment, theprogram is within the present invention. When all processing operationsof the supplied program or a part thereof is carried out by a functionexpansion board or a function expansion unit of the computer to realizethe functions of the digital camera (image pickup apparatus) 100 of thisembodiment, the program is within the present invention.

While the present invention has been described with reference toexemplary embodiment, it is to be understood that the invention is notlimited to the disclosed exemplary embodiment. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2008-074460, filed Mar. 21, 2008, which is hereby incorporated byreference herein in its entirety.

The invention claimed is:
 1. An image pickup apparatus having an imagepickup unit comprising: a first obtaining unit configured to obtain avisual image converted by the image pickup unit from an optical image ofan object formed by an optical unit; a second obtaining unit configuredto obtain measurement signals for measuring information corresponding todistance of the object, wherein the measurement signals are generated bythe image pickup unit converting optical images of the object intoelectric signals; a calculating unit configured to calculate atwo-dimensional distribution corresponding to the distance of the objectin the visual image by measuring the information using the measurementsignals; a storing unit configured to store light distributioncharacteristic data regarding light distribution characteristics of alight emitting unit; and a correction unit configured to correct thevisual image on the basis of the information included in thetwo-dimensional distribution calculated by the calculating unit and thelight distribution characteristic data corresponding to each informationincluded in the two-dimensional distribution.
 2. The image pickupapparatus according to claim 1, wherein the correction unit corrects aplurality of areas of the visual image on each area basis in accordancewith the information included in the two-dimensional distribution ofeach area.
 3. The image pickup apparatus according to claim 1, whereinthe second obtaining unit obtains the measurement signals from electricsignals converted from the optical image of the object pupil-divided bythe optical unit.
 4. The image pickup apparatus according to claim 1,wherein the image pickup element includes a first converting unit forobtaining the visual image and a second converting unit for obtainingthe measurement signals, each having pixels two-dimensionally arrangedon the same plane.
 5. The image pickup apparatus according to claim 1,wherein the correction unit corrects the visual image by taking, as thelight distribution characteristic data, a zoom position of the opticalunit into consideration in addition to the information.
 6. The imagepickup apparatus according to claim 1, further comprising an iris foradjusting an amount of light incident on the optical unit, wherein thecorrection unit obtains a degree of in-focus with respect to eachdistance included in the two-dimensional distribution by using a zoomposition and a focus position of the optical unit and an iris value ofthe iris, and corrects the visual image according to the degree ofin-focus.
 7. The image pickup apparatus according to claim 6, whereinthe correction unit corrects only an image area of a degree of in-focusequal to or more than a predetermined threshold value among the degreeof in-focus obtained for each image area.
 8. The image pickup apparatusaccording to claim 1, wherein the correction unit corrects only an imagearea within the distance of the object equal to or less than apredetermined threshold value among the distances included in thetwo-dimensional distribution calculated by the calculating unit.
 9. Theimage pickup apparatus according to claim 1, further comprising an inputunit which enters a photographing mode regarding photographing of theobject, wherein the correction unit determines whether to correct thevisual image according to the photographing mode entered with the inputunit.
 10. The image pickup apparatus according to claim 9, wherein theinput unit allows an operator to designate whether to correct the visualimage for each photographing mode.
 11. The image pickup apparatusaccording to claim 1, further comprising an input unit which enters atype of the optical unit, wherein the correction unit determines whetherto correct the visual image according to the type of the optical unitentered with the input unit.
 12. The image pickup apparatus according toclaim 11, wherein the input unit allows an operator to specify whetherto correct the visual image for each type of the optical unit.
 13. Theimage pickup apparatus according to claim 1, wherein the calculatingunit calculates the two-dimensional distribution in the visual image ona predetermined distance basis on the basis of the distance from themeasurement signals.
 14. A control method for an image pickup apparatusincluding an image sensor and a storing unit which stores lightdistribution characteristic data regarding light distributioncharacteristics of a light emitting unit, the control method comprising:a first obtaining step of obtaining a visual image converted by theimage sensor from an optical image of an object formed by an opticalunit; a second obtaining step of obtaining measurement signals formeasuring information corresponding to distance of the object, whereinthe measurement signals are generated by the image sensor convertingoptical images of the object into electric signals; a calculating stepof calculating a two-dimensional distribution corresponding to thedistance of the object in the visual image by measuring the informationusing the measurement signals; and a correction step of correcting thevisual image on the basis of the information included in thetwo-dimensional distribution calculated in the calculating step and thelight distribution characteristic data corresponding to each informationincluded in the two-dimensional distribution.
 15. The non-transitorycomputer-readable storage medium storing program code for causing acomputer to execute the control method according to claim
 14. 16. Animage pickup apparatus having an image pickup unit, comprising: a firstobtaining unit configured to obtain a visual image converted by theimage pickup unit from an optical image of an object formed by anoptical unit; a second obtaining unit configured to obtain measurementsignals for measuring information corresponding to distance of theobject, wherein the measurement signals are generated by the imagepickup unit converting optical images of the object into electricsignals; a calculating unit configured to calculate a two-dimensionaldistribution corresponding to the distance of the object in the visualimage by measuring the information using the measurement signals; astoring unit configured to store light distribution characteristic dataregarding light distribution characteristics of a light emitting unit; acorrection unit configured to correct the visual image on the basis ofthe information included in the two-dimensional distribution calculatedby the calculating unit and the light distribution characteristic datacorresponding to each information included in the two-dimensionaldistribution; and an input unit which enters a type of the optical unit,wherein the correction unit determines whether to correct the visualimage according to the type of the optical unit entered with the inputunit.
 17. An image pickup apparatus comprising: an image pickup unitwhich converts optical images of an object formed by an optical unitinto electric signals; an obtaining unit which obtains a visual imageregarding photographing of the object from the electric signals, whereinthe image pickup unit and the obtaining unit are disposed on a sameimage sensor; a calculating unit which calculates a two-dimensionaldistribution corresponding to distance of the object in the visual imageby measuring information corresponding to the distance of the objectusing the electric signals converted from optical images whichrespectively pass through different pupils; a storing unit which storeslight distribution characteristics data regarding light distributioncharacteristics of a light emitting unit; and a correction unit whichcorrects the visual image on the basis of the information included inthe two-dimensional distribution calculated by the calculating unit andthe light distribution characteristic data corresponding to eachinformation included in the two-dimensional distribution.
 18. An imageprocessing apparatus comprising: a first obtaining unit configured toobtain a visual image converted by an image pickup unit from an opticalimage of an object formed by an optical unit; a second obtaining unitconfigured to obtain measurement signals for measuring informationcorresponding to distance of the object, wherein the measurement signalsare generated by the image pickup unit converting optical images of theobject into electric signals; a calculating unit configured to calculatea two-dimensional distribution corresponding to the distance of theobject in the visual image by measuring the information using themeasurement signals; a storing unit configured to store lightdistribution characteristic data regarding light distributioncharacteristics of a light emitting unit; and a correction unitconfigured to correct the visual image on the basis of the informationincluded in the two-dimensional distribution calculated by thecalculating unit and the light distribution characteristic datacorresponding to information included in the two-dimensionaldistribution.
 19. The image processing apparatus according to claim 18,wherein the correction unit corrects a plurality of areas of the visualimage on an each area basis in accordance with the information includedon the two-dimensional distribution of each area.
 20. The image pickupapparatus according to claim 1, wherein the calculating unit measures adefocus amount between the measurement signals.
 21. The image pickupapparatus according to claim 1, wherein the calculating unit measures aphase difference between the measurement signals.
 22. The imageprocessing apparatus according to claim 18, wherein the calculating unitmeasures a defocus amount between the measurement signals.
 23. The imageprocessing apparatus according to claim 18, wherein the calculating unitmeasures a phase difference between the measurement signals.